Papers with neural architecture
Copied to clipboard
| Challenge: | a recent study shows that large pre-trained language models are not sufficient for Hebrew. |
| Approach: | They propose a large pre-trained language model for Hebrew that recovers morphological segments encoded in contextualized embedding vectors. |
| Outcome: | The proposed model obtains state-of-the-art on all tasks beyond contemporary Hebrew baselines. |
Copied to clipboard
| Challenge: | Lexicalized Tree Adjoining Grammars are a linguistically motivated grammar formalism that allows parsers to express linguistic generalizations that are not captured by statistical parsing. |
| Approach: | They propose a supertagging approach combined with deep learning to extract LTAG supertags from the French Treebank and propose n-best supertailing for German and French. |
| Outcome: | The proposed supertagging approach is able to extract LTAG supertags from the French Treebank and n-best supertracking for German and German. |
Copied to clipboard
| Challenge: | Prior work on argumentation in the NLP community has focused mainly on the first goal and has missed more nuanced and complex details of viewpoints. |
| Approach: | They propose a neural architecture that explicitly models the interplay between an Opinion Holder's (OH's) reasoning and a challenger's argument to predict if the argument succeeded in altering the OH' s view. |
| Outcome: | The proposed model outperforms several baselines on discussions on the Change My View forum on Reddit. |
Copied to clipboard
| Challenge: | Existing tools that support only a few major languages are under-optimized for accuracy due to a focus on efficiency or use of less powerful models. |
| Approach: | They introduce a Python natural language processing toolkit that supports 66 languages . they train Stanza on 112 datasets and show it generalizes well on all languages compared to other tools . |
| Outcome: | The proposed toolkit performs well on 112 datasets and is compatible with the popular Java CoreNLP software. |
Copied to clipboard
| Challenge: | Existing sentiment analysis models lack temporal information to capture semantics of long texts. |
| Approach: | They propose a framework to exploit task-related discourse structures for sentiment analysis. |
| Outcome: | The proposed framework improves the performance even beyond existing approaches based on human annotated data. |
Copied to clipboard
| Challenge: | a task-oriented dialogue system needs high conversational capability and can be easily adaptable to changing situations. |
| Approach: | They propose a retrieval-based conversational model that can rank a large set of responses . they propose supervised training and fine-tuning on limited data collected through a human-in-the-loop platform . |
| Outcome: | The proposed model can scale to rank a large set of responses in real-world situations. |
Copied to clipboard
| Challenge: | Existing efforts to capture event argument interactions are limited by the argument role type information of contextual entities. |
| Approach: | They propose to capture event argument interactions as a Seq2Seq-like learning problem where a sentence with a specific event trigger is mapped to a sequence of event argument roles. |
| Outcome: | The proposed neural architecture generates argument roles by incorporating contextual entities’ argument role predictions, like a word-by-word text generation process, thereby distinguishing implicit argument distribution patterns within an event more accurately. |
Copied to clipboard
| Challenge: | Existing methods for duplicate classification require manual review and assigning bugs to the correct teams. |
| Approach: | They propose a loss function that can detect duplicate bug reports and aggregate them into latent topics without supervision. |
| Outcome: | The proposed model outperforms state-of-the-art methods for duplicate classification on both cases and can learn meaningful latent clusters without supervision. |
Copied to clipboard
| Challenge: | Existing methods to learn mappings in the embedding space of text autoencoders are limited to a single-vector embeddment, which limits how much information can be retained. |
| Approach: | They propose a method to learn mappings in the embedding space of an autoencoder by extending it to Bag-of-Vectors Autoencodeurs (BoV-AEs) this allows to encode and reconstruct much longer texts than standard autoencodings . |
| Outcome: | The proposed method performs better than a standard autoencoder on unsupervised sentiment transfer. |
Copied to clipboard
| Challenge: | Dialogue Act tagging with ISO 24617-2 standard is a difficult task that requires multiple labels covering semantic, syntactic and pragmatic aspects of dialogue. |
| Approach: | They propose a neural architecture to increase Dialogue Act tagging accuracy by using low-frequency fine-grained tags. |
| Outcome: | The proposed model achieves state-of-the-art tagging results on DialogBank data set . it uses syntactic information in the form of Part-Of-Speech and dependency tags . |
Copied to clipboard
| Challenge: | Argumentation mining (AM) is a new field of research that uses dependency parsing to analyse arguments. |
| Approach: | They propose a neural end-to-end approach to argument mining based on dependency parsing . their model is biaffine dependency parsed and outperforms the current state-of-the-art . |
| Outcome: | The proposed model outperforms the state-of-the-art in component identification and relation identification. |
Copied to clipboard
| Challenge: | In short news articles, authors add exaggerations or fabricate events to manipulate readers' emotions. |
| Approach: | They propose to model the flow of affective information in fake news articles using a neural architecture and combine topic and affective data extracted from text. |
| Outcome: | The proposed model outperforms state-of-the-art methods on four real-world datasets and shows that it can capture the flow of affective information in fake news articles. |
Copied to clipboard
| Challenge: | Entity recognition is a widely benchmarked task in natural language processing . a neural architecture called BiLSTM-CRF is used to model the language sequences . |
| Approach: | They propose a neural architecture called BiLSTM-CRF to model the language sequences. |
| Outcome: | The proposed system achieves state-of-the-art on English entity recognition task and also in other languages. |
Copied to clipboard
| Challenge: | Experimental results show that the proposed method is significantly better than the baselines including the one solely based on BERT. |
| Approach: | They propose a neural architecture which uses a network for error detection and a system for error correction based on BERT, with the latter connected to the other using what they call soft-masking technique. |
| Outcome: | The proposed method performs better than baselines including the one solely based on BERT, and is general and may be employed in other language detection-correction problems. |
Copied to clipboard
| Challenge: | Reading comprehension (RC) is a high-level task in natural language understanding that requires reading a document and answering questions about its content. |
| Approach: | They propose to provide a standard neural network for reading a document and answering a question about its content. |
| Outcome: | The proposed model improves on the competitive SQuAD dataset by providing rich contextualized word representations and allowing it to choose between context-dependent and context-independent representations. |
Copied to clipboard
| Challenge: | Medical coding is time-consuming, expensive, and error prone. |
| Approach: | They propose to use diagnosis descriptions (DDs) of a patient as inputs to select the most relevant ICD codes. |
| Outcome: | The proposed algorithms perform on a clinical dataset with 59K patient visits. |
Copied to clipboard
| Challenge: | Existing word embeddings that use supervision only improve the embeddable word embeds of words with annotated lexical relations. |
| Approach: | They propose a supervised directional similarity network for learning task-specific transformation functions on top of general-purpose word embeddings. |
| Outcome: | The proposed model outperforms existing models on the HyperLex dataset on a directional graded lexical entailment task by 25%. |
Copied to clipboard
| Challenge: | asynchronous domains lack large labeled datasets to train an effective speech act recognition model. |
| Approach: | They propose methods to leverage abundant unlabeled conversational data and available labeled data from synchronous domains to train an effective SAR model. |
| Outcome: | The proposed method outperforms existing methods when trained on in-domain data only. |
Copied to clipboard
| Challenge: | Named entity recognition (NER) is a fundamental natural language understanding task that requires large amounts of high quality annotated in-domain data. |
| Approach: | They propose a neural architecture that leverages the semantic information in the names of the labels to give the model additional signal and enriched priors. |
| Outcome: | The proposed model is especially effective in low resource settings. |
Copied to clipboard
| Challenge: | Existing work on learning contextual representations has used LSTM-based biLMs, but there is no reason to believe this is effective. |
| Approach: | They propose to use pre-trained bidirectional language models to learn contextual word embeddings for four NLP tasks and to use them to study the effects of architecture on endtask accuracy. |
| Outcome: | The proposed models outperform word embeddings for four NLP tasks and all learn representations that vary with network depth. |
Copied to clipboard
| Challenge: | Numeracy is the ability to understand and work with numbers. |
| Approach: | They propose a neural architecture that uses a continuous probability density function to model numerals from an open vocabulary using hierarchical models. |
| Outcome: | The proposed model reduces errors by 18% and 54% on clinical and scientific datasets compared to the second best model for each dataset . |
Copied to clipboard
| Challenge: | Recent neural architectures such as Transformer and BERT allow for multimodal context, which may help model the meaning of words in general and also sharpen its understanding of instances of words. |
| Approach: | They propose a neural architecture that combines vision and natural language to advance models of multimodal context. |
| Outcome: | The proposed architecture achieves the highest performance on the Visual Commonsense Reasoning benchmark and the best performance to date on the public leaderboard. |
Copied to clipboard
| Challenge: | a relatively small fine-tuned encoder-decoder model performs better than out-of-the-box LLMs and semantic role labeling parsers to accurately predict the relevant scenario information. |
| Approach: | They propose a neural architecture finetuned for the task of scenario context generation . they use a curated dataset of time and location annotations to train an encoder-decoder architecture . |
| Outcome: | The proposed model performs better than out-of-the-box LLMs and semantic role labeling parsers to accurately predict the relevant scenario information of a particular entity or event. |
Copied to clipboard
| Challenge: | Experimental results show that a neural architecture that combines both modalities yields better results. |
| Approach: | They propose a neural architecture that combines both modalities to solve the problem of determining whether people are located in tweets. |
| Outcome: | The proposed model combines both modalities to produce better results . |
Copied to clipboard
| Challenge: | Recent studies have focused on English language and tasks, but few have explored the complexity of a SLU task. |
| Approach: | They propose to explore Neural Networks approaches for a French Spoken Language Understanding task. |
| Outcome: | The proposed approach outperforms classical Neural Network Architectures and achieves state-of-the-art results. |
Copied to clipboard
| Challenge: | Social media based micro-blogging sites like Twitter are used for expressing emotions and opinions. |
| Approach: | They propose to combine convolutional and fully connected layers in a non-sequential manner to train deep multi-task learning models trained for all emotions at once in unified architecture. |
| Outcome: | The proposed model outperforms the previous system by 0.044 or 4.4% on the WASSA’17 EmoInt shared task dataset. |
Copied to clipboard
| Challenge: | Identifying the political perspective shaping the way news events are discussed in the media is an important and challenging task. |
| Approach: | They propose a neural architecture for representing relational information to capture social context of news documents. |
| Outcome: | The proposed model performs better than supervised models in the supervised setting and shows that it provides a distant supervision signal. |
Copied to clipboard
| Challenge: | We show that one of today’s largest language models lacks this kind of social intelligence out-of-the-box, using two tasks: SocialIQa and ToMi. |
| Approach: | They propose to use social intelligence and Theory of Mind to examine whether modern large-scale language models lack this kind of social intelligence out-of-the-box. |
| Outcome: | The proposed model lacks social intelligence out-of-the-box, and has well-below human accuracies on SocialIQa and ToMi, respectively. |
Copied to clipboard
| Challenge: | Using a joint approach, we found that the model is more efficient than those developed in machine reading (MR) work. |
| Approach: | They propose a joint model for selecting correct answer sentences among the top k provided by answer sentence selection modules. |
| Outcome: | The proposed model improves on WikiQA, TREC-QA, and a real-world dataset. |
Copied to clipboard
| Challenge: | Neural architectures are the current state of the art in Word Sense Disambiguation (WSD) however, they make limited use of the vast amount of relational information encoded in Lexical Knowledge Bases (LKBs). |
| Approach: | They propose a neural supervised architecture that embeds Lexical Knowledge Bases and exploits pretrained synset embeddings to predict synsets that are not in the training set. |
| Outcome: | The proposed architecture breaks through the 80% ceiling on the concatenation of all standard all-words English evaluation benchmarks. |
Copied to clipboard
| Challenge: | Metaphor is a linguistic device in which a concept is expressed by mentioning another . Verbal MWEs are examples of non-literal language in which multiple words form a single unit of meaning. |
| Approach: | They propose to analyze the interplay between metaphor and multiword expressions processing by informing the model of the presence of MWEs. |
| Outcome: | The proposed architecture reach state-of-the-art on two established metaphor datasets. |
Copied to clipboard
| Challenge: | Predicting Machine Translation (MT) quality has been limited to word and sentence-level prediction. |
| Approach: | They propose a framework that can generalize neural QE approaches to the level of documents. |
| Outcome: | The proposed framework outperforms state-of-the-art approaches on document-level quality estimates and is 40 times faster to train. |
Copied to clipboard
| Challenge: | Existing models that assume static user interests are unable to capture the temporal aspects of user interactions and interest changes over time. |
| Approach: | They propose a neural architecture to exploit changes of user interactions and interests over time to predict which discussions they are likely to enter. |
| Outcome: | The proposed model outperforms state-of-the-art models that assume static user interests and handle future conversations that are unseen during training time. |
Copied to clipboard
| Challenge: | Argumentation is the process by which arguments are constructed, compared, evaluated in several respects and judged in order to establish whether any of them is warranted. |
| Approach: | They propose to annotate 1908 arguments tagged with quality facets from a resource of 402 persuasive essays and to use them to create a neural architecture that takes into account the support and attack relations holding among the arguments. |
| Outcome: | The proposed neural architecture outperforms state-of-the-art and standard arguments on the persuasive essays dataset. |
Copied to clipboard
| Challenge: | Existing approaches to incorporate bilingual dictionaries into Neural Machine Translation (NMT) models have been criticized for lack of integration of bilingual lexical information into the neural architecture. |
| Approach: | They propose a neural architecture to incorporate bilingual dictionaries into Neural Machine Translation models by introducing three new components: Pointer, Disambiguator, and Copier. |
| Outcome: | The proposed method achieves the following merits inherently compared with previous efforts: (1) Pointer leverages the semantic information from bilingual dictionaries, for the first time, to better locate source words whose translation in dictionary can potentially be used; (2) Disambiguator synthesizes contextual information from source view and target view, both of which contribute to distinguishing translation of a specific source word from multiple candidates in dicaries; (3) Copier systematically connects Pointer and Disambiguators based on a hierarchical |
Copied to clipboard
| Challenge: | Recent advances in NMT have shown promising results but are vulnerable to noise. |
| Approach: | They propose a data-driven technique called Target Augmented Fine-tuning to incorporate noise during training. |
| Outcome: | The proposed techniques perform with no degradation where up to 10% of entire test words are infected by noise. |
Copied to clipboard
| Challenge: | Existing QA systems for question answering are limited by the availability of annotated datasets. |
| Approach: | They propose a dataset for question-answering that extracts information from multiple parts of text . they propose QA-based multi-span neural architecture that captures relevance among multiple answer spans . |
| Outcome: | The proposed model outperforms state-of-the-art QA models in this multi-span QA setting. |
Copied to clipboard
| Challenge: | Existing sentiment lexicons do not handle word sense and the concept of semantic compositionality is non-existent in simple lexiconic approaches. |
| Approach: | They propose a lexicon-driven contextual attention mechanism and a contrastive co-attention mechanism that models contrasting polarities between all positive and negative words in a sentence. |
| Outcome: | The proposed model outperforms many other neural baselines on sentiment classification tasks on multiple benchmark datasets. |
Copied to clipboard
| Challenge: | Low-resource language name tagging is an important but challenging task. |
| Approach: | They propose a neural architecture that leverages multi-level adversarial transfer to improve name tagging for low-resource languages. |
| Outcome: | The proposed approach outperforms previous approaches on CoNLL data sets. |
Copied to clipboard
| Challenge: | Existing approaches to Automated Essay Scoring (AES) treat scoring and feedback as separate components, resulting in fragmentation. |
| Approach: | They propose a psychometrically-aware framework that integrates diagnostic assessment with instructional scaffolding through a shared latent ability representation. |
| Outcome: | The proposed framework integrates diagnostic assessment with instructional scaffolding through a shared latent ability representation. |
Copied to clipboard
| Challenge: | Recognizing that various textual spans across multiple texts refer to the same entity or event is an important NLP task. |
| Approach: | They propose a neural architecture for cross-document coreference resolution by representing an event mention using its lexical span, surrounding context, and relation to other mentions via predicate-arguments structures. |
| Outcome: | The proposed model outperforms the state-of-the-art event coreference model on ECB+ while providing the first entity coreference results on this corpus. |
Copied to clipboard
| Challenge: | Automated essay scoring (AES) is a task of assigning a single score to an essay . authors abandon sophisticated neural architectures and develop a simple feature-based approach . |
| Approach: | a team of researchers develop a feature-based approach to cross-prompt automated essay scoring that adopts a simple neural architecture. |
| Outcome: | a new approach to cross-prompt automated essay scoring can achieve state-of-the-art results. |
Copied to clipboard
| Challenge: | Existing approaches to zero-shot cross-lingual transfer have focused on training with adapters of a single source and testing either with the target LA or LA of another related language. |
| Approach: | They propose to leverage LAs of multiple (linguistically or geographically related) source languages for more effective cross-lingual transfer instead of just one source LA . they extend their novel neural architecture, ZGUL, to settings where either (1) some unlabeled data or (2) few-shot training examples are available for the target language . |
| Outcome: | Extensive experimentation across four language groups, covering 15 unseen target languages, shows improvements of up to 3.2 average F1 points over baselines on POS tagging and NER tasks. |
Copied to clipboard
| Challenge: | Existing methods for named entity recognition focus on augmenting in-domain data in low-resource scenarios where annotated data is limited. |
| Approach: | They propose a neural architecture to transform data from high-resource to low-resourced domains by learning the patterns in the text that differentiate them. |
| Outcome: | The proposed approach improves on high-resource domain representations over high- and low-resourced domains. |
Copied to clipboard
| Challenge: | Existing models for temporal ordering of events rely on pretrained representations, transfer and multitask learning, and self-training techniques. |
| Approach: | They propose a neural architecture and a set of training methods for ordering events by predicting temporal relations by pre-training models. |
| Outcome: | The proposed models can predict temporal relations between two pairs of events within a span of text and identify temporal relationships between them. |
Copied to clipboard
| Challenge: | Xu et al., 2016) show that a simple neural architecture can be efficiently used for in-domain and cross-domain text simplification. |
| Approach: | They evaluate neural sequence-to-sequence models for text simplification on Wikipedia and Newsela datasets. |
| Outcome: | The proposed model can generalize across corpora and overcome challenges when tested on Wikipedia and Newsela datasets. |
Copied to clipboard
| Challenge: | We train a 170Mparameter Backpack language model on OpenWebText, matching the loss of a 6Bparameter Transformer. |
| Approach: | They propose a neural architecture that learns multiple non-contextual sense vectors for each word in a vocabulary and represents a word as a context-dependent, non-negative linear combination of sense vector. |
| Outcome: | The proposed model outperforms a GPT-2's word embeddings on lexical similarity evaluations and can be used to perform controllable text generation and debiasing. |
Copied to clipboard
| Challenge: | Recent neural methods for keyphrase extraction are mostly observed in documents originating from the scientific domain. |
| Approach: | They develop a neural keyphrase extraction model that goes beyond language understanding to handle the variations of domain and content quality. |
| Outcome: | The proposed model can handle the variations of domain and content quality without restriction of the domain, quality, nor content of the documents. |
Copied to clipboard
| Challenge: | Recent research indicates that these datasets are trivially annotated to a large degree. |
| Approach: | They propose to use a trending NLP task to test a neural architecture for NLI . they show a pathological sensitivity of the systems to permutations of the training set . |
| Outcome: | The proposed model outperforms the baseline model by multiple percentage points . the proposed model is based on the proposed dataset and the soundness of the task . |
Copied to clipboard
| Challenge: | a new neural architecture can be used to classify stances on social media without relying on linguistic features. |
| Approach: | They propose a neural architecture where the input also includes automatically generated negated perspectives over a given claim. |
| Outcome: | The proposed model improves on the original input and removes doubtful predictions over the retained information. |
Copied to clipboard
| Challenge: | Recent state-of-the-art neural models have obtained F1-scores near 98% on the task of slot filling. |
| Approach: | They propose to fix annotation errors in ATIS and propose a rule-based grammar for slot filling that achieves a 95.82% F1 score. |
| Outcome: | The proposed grammar achieves a 95.82% F1-score on the ATIS domain. |
Copied to clipboard
| Challenge: | Existing studies on Arabic sentiment analysis (SA) tasks focus on word embeddings to capture semantic and syntactic similarities, but Arabic language is characterized by its agglutination and morphological richness contributing to great sparsity. |
| Approach: | They propose several protocols to evaluate specific embeddings for Arabic sentiment analysis task. |
| Outcome: | The proposed embeddings are based on words and lemmas in Arabic sentiment analysis (SA) task. |
Copied to clipboard
| Challenge: | a neural architecture learns to generate content in a specific order without explicit specifications of the relations between input entities and output entities. |
| Approach: | They propose a natural language generation task that generates discharge instructions from ICD codes . they propose to model content ordering and text generation in a specific order . |
| Outcome: | The proposed model outperforms baseline models in BLEU scores and human evaluation. |
Copied to clipboard
| Challenge: | Named entity recognition models are challenging for languages with little training data. |
| Approach: | They propose a simple and efficient neural architecture for cross-lingual named entity recognition models. |
| Outcome: | The proposed model achieves competitive performance with the state-of-the-art on two transferable factors: sequential order and multilingual embedding. |
Copied to clipboard
| Challenge: | Existing work on map-based NL navigation relies on small artificial worlds with a fixed set of entities known in advance. |
| Approach: | They propose a task to interpret navigation instructions in natural language (NL) they use a dataset aligned with real, dense, urban maps to study neural architectures . |
| Outcome: | The proposed task is based on a dataset of 2515 navigation instructions aligned with real routes over three regions of Manhattan. |
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) have stimulated a surge of research aimed at extending their applications to the visual domain. |
| Approach: | They propose a novel neural architecture to enhance language-image understanding capability of LLMs by capturing intricate details that are often overlooked in existing methods. |
| Outcome: | The proposed model can achieve better comprehension of language information in visual contexts within images. |
Copied to clipboard
| Challenge: | In-Image Machine Translation (IIMT) aims to convert images containing texts from one language to another. |
| Approach: | They propose an end-to-end model instead of the traditional cascade methods which use optical character recognition followed by neural machine translation and text rendering. |
| Outcome: | The proposed model outperforms both cascade methods and current model in translation quality and robustness across various dimensions. |
Copied to clipboard
| Challenge: | Existing studies show that global and local attention are expressively complementary. |
| Approach: | They propose to restrict global attention to a fixed-size window of preceding tokens . they also propose to add local attention to local-only transformers to increase model quality . |
| Outcome: | The proposed model outperforms the global–local transformers on formal language recognition and natural language modeling. |